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Abstract 

We show a new convergence theorem for infinite products of stochastic matrices. It generalizes the 
popular condition of a positive diagonal of all factors to the existence of what we call an aperiodic 
core. An aperiodic core is a common aperiodic digraph without sinks that is a subgraph of the 
digraph described by the factor matrices. Our result generalizes both a theorem of Wolfowitz and 
1 convergence theorems in the application domain of asymptotic consensus. 

^ '■ 1 Introduction 

A stochastic matrix is a real matrix with nonnegative entries whose row sums are equal to 1 . Long and 
infinite products of stochastic matrices have found numerous applications, in particular in computer 
science for multi-agent systems whose agents start with a private value and repeatedly form averages 
of perceived values of others. These types of multi-agent systems are not only used in computer 
■ networks, but have also been found to model various physical and biological phenomena like the 

. ' behavior of bird flocks [I] [3] • 

, In these application, it is of interest to know under which conditions an infinite product of 

stochastic matrices converges. Also of interest are conditions under which the limiting matrix has 
rank 1, for this ensures asymptotic agreement amongst the agents. 

The first convergence result for products of stochastic matrices is the Perron-Frobenius theorem, 
which states that the powers of an ergodic stochastic matrix converge to a rank I stochastic matrix. 
Ergodicity here is equivalent to the fact that the matrix has a power that is positive, i.e., all of 
its entries are positive. This result was first generalized to a non-constant product of matrices by 
Wolfowitz [6j: 

00 ■ 

^ ' Theorem 1 (Wolfowitz, 1963). Let M be a finite set of stochastic n x n matrices with the property 

t that every finite product of matrices in M is ergodic, and let A(k) be a sequence of elements of Ml. 

Then the sequence of products A(k) ■ A(k — 1) ■ ■ ■ A(l) converges to a rank 1 stochastic matrix. 

The strict finiteness and ergodicity conditions in Wolfowitz' theorem were found to be inappro- 
£C) • priate for the mentioned applications. Subsequently, Theorem fj] was extended in several directions 

(see, for example, [1] Theorem 2] or [3] Section II. G]). However, no direct generalization of Wol- 
fowitz' theorem was obtained. These results required a constant lower bound on positive matrix 
entries and an eventual positivity assumption in the product. These conditions are also fulfilled in 
Wolfowitz theorem. Additionally, they required a positive diagonal and pseudo-symmetry of the 
matrices. (More specifically, a pseudo-symmetric matrix A is one such that Aij is positive if and 
only if Aj,i is.) 

These results were recently generalized by Touri and Nedic [4] as part of a result on random 
products of stochastic matrices. They replaced pseudo-symmetry by the more general property of 
complete reducibility. A matrix is completely reducible if it is not permutation similar to a matrix of 
the form 

f A 



v G J 

where both A and C are quadratic and B 7^ 0. A non-probabilistic version of their theorem is the 
following: 

Theorem 2 (Touri and Nedic, 2011). Let A(k) be a sequence of n x n stochastic matrices with the 
following properties: 

1. Positive entries are lower bounded, i.e., there exists an a > such that Aij(k) > implies 
Aij(k) ^ a for all i, j, and k. 



1 



2. Eventual positivity: 

For every k there exists a K ^ k such that the matrix Y^,k'=k A[k') -A(k' — 1) ■ ■ ■ A(k) is positive. 

3. Every A(k) is completely reducible. 

4- All diagonal entries Ai,i{k] are positive. 

Then the sequence of products A(k) ■ A(k — 1) ■ ■ ■ converges to a stochastic matrix of rank 1. 

If the eventual positivity condition (2) is omitted, convergence still holds, but the limit is not 
guaranteed to have rank 1. There also exists an analogous variant of our main theorem when this 
condition is dropped. 

The purpose of this paper is to generalize Theorem [2] by replacing condition (4) of positive 
diagonal entries by the more general requirement for the existence of an aperiodic core. The notion 
of an aperiodic core is a graph-theoretical one which we detail now: 

To every stochastic nx n matrix A, we assign the digraph G(A) with node set [n] = {1, 2, . . . , n} 
containing an edge (i,j) if and only if the matrix entry Aij is positive. A matrix is completely 
reducible if and only if its digraph is does not contain edges between its strongly connected compo- 
nents. A strongly connected digraph is aperiodic if the lengths of the cycles it contains are coprime. 
A not strongly connected digraph is aperiodic if all its strongly connected components are. Clearly, 
a digraph containing self-loops at all nodes is aperiodic. This corresponds to the case of a positive 
diagonal. We will use Wielandt's theorem [5] on the existence of walks with prescribed lengths in 
aperiodic digraphs to replace the self-loops, which have an exponent equal to zero, by an arbitrary 
aperiodic core. 

The following theorem is the main result of this paper. It generalizes at the same time Theorem[T] 
and Theorem A sink is a node without outgoing edges. 

Theorem 3. Let A(k) be a sequence of n x n stochastic matrices with the following properties: 

1. Positive entries are lower bounded. 

2. Eventual positivity: 

For every k there exists a K k such that the matrix Yly=k A(k') -A(k' — 1) ■ ■ ■ A(k) is positive. 

3. Every A(k) is completely reducible. 

4- The sequence has an aperiodic core, i.e., there exists an aperiodic digraph H with node set [n] 
and without sinks such that H is a subgraph of every G(A(k)) . 

Then the sequence of products A(k) ■ A(k — 1) • •• ^4(1) converges to a stochastic matrix of rank 1. 

The rest of this paper is devoted to the proof of Theorem [3] 

2 Preliminaries 

This section contains the basic tools for our convergence proof. On one hand, this is a semi-norm 
for stochastic matrices. We show that if the semi-norm of a product converges to zero, then firstly 
the product converges and secondly the limit matrix is stochastic of rank 1. On the other hand, 
we introduce a graph-theoretic interpretation of matrix products and present Wielandt's theorem, 
which we will use to show positivity of elements in partial products. 
For denoting partial products, we introduce the notation 

P(k) = A(k) ■ A(k - 1) ■ ■ ■ A(l) 

and 

P(k, I) = A(k) ■ A(k - 1) • • • A(l + 1) . 
In particular, P(k) = P(k, I) ■ P(l) whenever I ^ k. 

2.1 Matrix Semi-Norm for Stochastic Matrices 

All stochastic matrices have 1 as an eigenvalue of maximum modulus. If the matrix is irreducible, 
the corresponding right-eigenspace is one-dimensional and generated by the column vector 1 = 
*(1, 1, . . . , 1). When studying such matrices, we are hence led to consider the distance of vectors to 
this eigenspace. Indeed, we will see in Lemma[5]that considering this distance is an appropriate tool 
for products of stochastic matrices. This technique for proving convergence results for products of 
stochastic matrices was communicated to the author by Charron-Bost [2]. 

Starting from the vector semi-norm that measures the distance to the common eigenspace R ■ 1, 
we will define a corresponding matrix semi-norm in analogy to the operator norm. A preliminary 
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result that we need later on is the follwing simple lemma which shows that the operator norm of a 
stochastic matrix with respect to the oo-norm is at most 1: 

Lemma 1. If A is an n x n stochastic matrix, then \\Ax\\oo ^ \\ x \\oo for all x G R n . 
We define the vector semi-norm on R™ by setting 

||a;|| = inf \\x - j/||oo ■ 

This is indeed a semi-norm: The homogeneity property ||q ■ x\\ — \a\ ■ \\x\\ follows directly from 
homogeneity of the oo-norm. The triangle inequality also holds: Let 3:1,2:2 G R" and 2/1,2/2 G R • 1. 
The sum y — yi + 2/2 remains in the eigenspace R • 1 and, because of the infimum in the definition 
of \\xi + X2W and the triangle inequality for the oo-norm, we get 

Hasi + as2 1| < ||a:i + x 2 - j/||oo < ||aca - 2/i||<x> + ||» 2 - 2/a||<x> • 

Forming the infimum over all y\ and 2/2 now shows \\xi + x 2 \\ ^ + 11^2 1|- 

This vector semi- norm induces a matrix semi- norm on R nx " by defining it in the operator norm 
fashion: 

M " » W 

11*11 5*0 

This definition ensures the homogeneity property ||a-A|| = |a|-||j4.||, the triangle inequality ||A+S|| ^ 
||A|| + ||B||, and submultiplicativity \\A-B\\ < ||A||-||B||. Clearly, \\A\\ = if the image of A is contained 
in the subspace R ■ 1. 

The following lemma shows the utility of the matrix semi-norm to prove convergence of products 
of stochastic matrices. It is a core ingredient to our proof of Theorem [3] 

Lemma 2. If ||P(fe)|| — > 0, then the sequence of partial products P(k) converges and the limit is a 
stochastic matrix of rank 1. 

Proof. We show that, for every x G E", the sequence of vectors P(k) ■ x converges by showing that it 
is Cauchy. This then concludes the proof because stochasticity is preserved when taking limits and 
the matrix semi-norm is continuous, implying that the limit's semi-norm is zero. 

Let e > 0. Because also ||P(fc) ■ x\\ —¥ 0, there exists some K such that ||P(if) -x|| ^ e/2. Letting 
y g R • 1 such that || P(lf) • x|| = ||P(A") ■ x — y\\oo, we calculate for every k ^ K: 

\\P{k) ■ x - P{K) • xlloc < ||P(fe, K) ■ P(K) ■ x ~ 2/IU + \\P(K) ■ x - yWoo 
= ||P(fc, K) ■ (P(K) ■x-y)\\ oc + \\P(K) -x-yWvc 
< 2 • \\P(K) ■ x - 2/IU = 2 • \\P(K) • x\\ < e 

because P(k, K) is a stochastic matrix, which implies y = P(k, K)-y and that Lemma[T]is applicable. 
This shows that P(k) ■ x is a Cauchy sequence. □ 

We now provide a tool to prove convergence of the matrix semi-norm of a product to zero by 
stating a sufficient condition for the semi-norm of a factor to be constantly bounded away from 1. It 
shows in particular that the semi-norm of a stochastic matrix is at most 1. 

Lemma 3. Let A be an n x n stochastic matrix with Aij a for all i,j £ [n]. Then \\A\\ ^ 1 — na. 

Proof. Write A = E + B where E = (a)ij. Because the image of E is contained in R ■ 1, we 
have \\E\\ = and hence \\A\\ — \\B\\ by the triangle inequality. The entries of B are all nonnegative 
and its row sums 53 . Bi t j are all equal to 1 — na. 
Let x £ R n . For every c G R, we have: 

||.B2;|| ^ \\Bx — (1 — na)c ■ l||cx> = max j ( Bi t jXj) — (1 — na)c\ 

3 

= max I ^S~\ Bij ■ {xj — c)| ^ maxVj Bij ■ \xj — cj 

i 3 

^ max) Bi 4 • \\x — c ■ l||oo = (1 — na) ■ \\x — c • l||oo 

3 

Forming the infimum over all c shows ||Bx|| ^ (1 — na) ■ \\x\\ and hence ||A|| = ||B|| ^ 1 — na. □ 
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2.2 Graph Interpretation of Matrix Products 



Let i and j be nodes of a digraph G. A walk in G from i to j is a finite sequence of consecutive edges 
in G that starts at i and ends at j. Its length is the number of edges in the sequence. The empty 
sequence is a walk from i to j only if i = j. 

The following lemma characterizes positivity of entries in products of stochastic matrices solely 
in terms of the matrices' associated digraphs. Denote by K n the complete digraph with node set [n]. 

Lemma 4. Let ^ Z ^ k and i,j £ [n]. Then Pij(k,l) is positive if and only if there exists a walk 
(efc, efc_i, . . . , e;+i) of length k — I in K„ from i to j such that e m is an edge of G[A(m)) for all 
I + 1 ^ m ^ k. 

Proof. We proceed by induction on k — I. 

The case k = I is trivial because P(k, k) is the identity matrix: Pij(k, k) is positive if and only 
if i = j. But also, there exists a walk of length from i to j if and only if i = j. 

Let now k — I ^ 1. It is 

Pi* l )= (*) • P -'-.i (* - M) C 1 ) 

i'6[n] 

by definition of P(Zc, Z). 

First assume that Pij(k, I) is positive. Then, by Equation |T]), there exists some i' £ [n] such that 
both Ai t ii (k) and Pi> i j(k — 1, Z) are positive. By definition of G(A(Zc)) , it contains the edge eu = (i,i'). 
By the induction hypothesis, there exists a walk (e^-i, . . . , e;+i) in K n from i' to j such that e m is 
an edge of G[A(m)) for all I + I ^ m sj k — 1. Hence the extended walk (e^, e^-i, . . . , e; + i) fulfills 
the desired properties. 

Now assume the existence of a walk (e^, e^-x, . . . , e;+i) in K n from i to j such that e m is an edge 
of G( A(m)) for all Z + 1 m ^ fc. Let = (i, i'). By definition of G(A(fc)) , the entry A^^/ (k) is pos- 
itive. Also, by the induction hypothesis, the entry P t i j(k — 1, Z) is positive. Hence, by Equation ([T]), 
also Pij(k,l) is positive. □ 

If a strongly connected digraph is aperiodic, there exist walks of arbitrary length between all 
pairs of nodes as long as the length is greater or equal to a number called the exponent (sometimes 
also index) of the digraph. Wielandt [5] provided an upper bound on the exponent. It is the best 
possible upper bound in terms of only the number of nodes. 

Theorem 4 (Wielandt, 1950). Let G be a strongly connected aperiodic digraph with n nodes and 
let i and j be nodes of G. Then there exists a walk in G from i to j of length k whenever k is greater 
or equal to 

W(n) = n 2 - 2n + 2 . 

3 Proof of Theorem [3] 

Let a > be a lower bound on the positive matrix entries, i.e., Aij(k) ^ a whenever Aij(k) is 
positive. In this section, we show the existence of a K such that 

\\P{K)\\ 1 - na n{w{n)+1) . (2) 

This suffices to show the theorem because the right-hand side of Equation ([2]| is strictly less than 1 
and thus repeated application shows that ||P(fc)|| — > by submultiplicativity of the matrix semi- 
norm. Lemma[2]then concludes the proof. 

For every j £ [n], define Sj(k) to be the set of indices i G [n] such that Pij(k) is positive. We 
use Wielandt 's theorem to show that the sequences Sj(k) are almost nondecreasing: 

Lemma 5. Whenever k — I ^ Win), we have Sj(l) C Sj(k). 

Proof. Let i G Sj(l). Denote by H the strongly connected component of i in the aperiodic core H. 
Because H is aperiodic, Theorem [3] implies the existence of a closed walk of length k — I at node i 
in H. The fact that H is a subgraph of all G(j4(m)) shows that Pi,i{k, I) is positive by LemmaU 
Hence, because Pijil) is positive and Pij(k) ^ Pi t i(k,l) ■ Pi,j(l), so is Pij(k). □ 

Denote by fij(k) the smallest (positive) Pij(k) where i £ Sj(k). It is not hard to see that 
fj-j(k) ^ a k ~ l ■ fij(l) whenever ^ Z ^ k. Under cerain conditions we even have a stronger inequality: 

Lemma 6. If Sj(k) = Sj(k + 1), then (ij(k + l) > fij(k). 
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Proof. Let Pi,j(k + 1) be positive, i.e., i G Sj(k + 1) = Sj(k). By definition of Sj(k), we have 

P itj (k + 1)= A t j(k + 1)- P (:J (k) . (3) 

Because Sj(k) = Sj(k + 1), we derive that A ii t(k + 1) is zero whenever i G" Sj(k) and t G Sj(k). Hence, 
because A(k) is completely irreducible, we also have that A it i(k + 1) is zero whenever i G Sj(k) and 

By assumption, we have i £ Sj(k), and hence by the above and by stochasticity of A(k + 1): 

l=^A M (fe + l)= ^ + (4) 

Because Pej(k) Mj(fe) for all ^ G Sj(k), combination of Equations (J3J) and Q yields Pij{k + 1) ^ 

Together with Lemma[3] the next lemma shows Equation |2|. concluding the proof of Theorem[3] 
Lemma 7. There exists a K such that Pij(k) > Q n ( lv ( n )+ 1 ) y or all i,j G [n] and all k ^ K. 

Proof. Let j G [n]. For every i G [n], let fcj be the least nonnegative integer such that i G Sj(ki). 
Because of the eventual positivity property (2) in the theorem statement, all ki are well-defined. By 
permutating indices, we can assume without loss of generality that fci ^ fe ^ ■ ■ ■ ^ k n . Because P(0) 
is the identity matrix, we have Sj(0) = {j} and hence k\ — 0. 
We inductively show 

> « (m " 1)(, " (n)+1) (5) 

for all 1 ^ m ^ n. This is true for m = 1. To prove the inductive step, we distinguish two cases: 
(A) k m - fc m _i < W(n) and (B) k m - fc m _i ^ W(n). 
In case (A), we have 

by the induction hypothesis. 

In case (B), we have S 3 (k) — Sj (fc m -_i) for all k with k m - \ +W(n) ^ k ^ fc m — 1 by Lemma[5]and 
the definition of k m . Repeated application of Lemma|6]hence yields Hj{k m — 1) ^ fij (k m - \ + W(n)j. 
We thus have 

Mj(fem) > a • fij(k m - 1) > a - /ij(fc m _i + W^n)) 

by the induction hypothesis. 

In particular, we have shown Equation (J5]l for m — n. Now set K — k„ + W (n). By Lemma [5] 
Sj(k) = [n] for all k ^ K. Also, ^(fc) > fj, S (K) > a 14 ''"' ■ /Lt f (fe) > Q n ( w C™)+ 1 ) by Lemma E] and 
Equation (|5j). This shows the entries of the jth column of P(k) are all greater or equal to a"'"''°' +1 ' 
whenever k ^ K. Because the choice of j was arbitrary, we have shown the lemma. □ 
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